Final Project - Data Analytics with R

1. Data Summary

The data is recieved from the Amercian Communinity Survey. The data basically consist of the personal information of the people currently residing in United States of America between 2012 - 2016. This is basically an observational study on over 10 Million of records of data being collected.

Here is a glimpse/description of data which is used to perform this analysis.

The Public Use Microdata Sample (PUMS) contains a sample of actual responses to the American Community Survey.The Public Use Microdata Sample (PUMS) contains a sample of actual responses to the American Community Survey (ACS). The PUMS dataset include variables for nearly every question on the survey. Each record in the file represents a single person.The PUMS contain data on approximately one percent of the United States population.

The data contains 7,487,361 rows and 234 columns. Each row this very data set is scribbled by a response from one person.

Since the data is quite large , I have filtered out only 8 cases which is required for my analysis. The important variables I have choosen for my analysis are as follows :-

Data Description

## Time difference of 1.140622 mins

2. Methodology

To perform the analysis on this I have taken the housing data and have covered all the major columns which actually give precise and detailed information about the housing information and patterns across the United States of America. Since this data comes with Property Information , Resources Consumption, and their expenditure information , I have majorily and explicitly focus on the analysis which yields the information related to Housing Scheme. In this project I have also covered the variablity of each economic variable and put the light on its usage and on its dominion on other important variables.

Also,this data comes with the demographics information which helps to predict the future outcomes of the usablity in terms of transportation , in terms of land price and I have also discussed about the amenities currently United States people make use of.

Since this data also consist of State Information so it was quite obligatory to discuss about the information with respect to the pattern of data across different states.I have unanimously compared the states in terms of the various statistics which eventually produce Jaw Dropping Results which I initially not expecting (Though I wonder) but comes with true results.

While following the Methodology , following things raised :-

How did you deal with missing values? What impact does your approach have on the interpretation or generalizability of the resulting analysis?

• Honsetly speaking, In sum of total, this data consist of more ‘NA’s rather than the actual information. Almost all the columns consist of the ’NA’ present in it which is quite obvious since its an actual and opinion data where information availability is not obvious all the time.

• To overcome this problem , I have removed the missing values (NA) from the variables that is involved in our analysis and stored the data into a individual dataframes. Dplyr library helps here to filter the data which is not useful and is not proximity of our usage.

• Removing the NA’s from the data not only produce the clean and tidy results but its also produce the valuable and precise results. Removing the NA from the data increase the importance of the column and I have observed the difference between the results with NA and the results without NA. For intance - NA always deviates the result from the actual results/outcome and this would increase the chance of what is called FALSE POSITIVE.

How did you deal with outliers?

• Since this data is actual information (opinion and option based) there are mere chances of having the outliers. Most of the outliers in this data are usually the NA which are present in almost every columns. To overcome this issue I first address whether this value is really an outlier that needs attention or its just the normal false positive.
• To deal with that I removed that from the data owing to the fact that outliers can drastically bias/change the fit estimates and predictions. However those were not much influential.

How did you deal with weights & income adjustment values?

• There is no standard way of addressing weights. While many R functions have a weights parameter, there is no consistency in how they are intepreted: Most commonly, weights in R are interpreted as frequency weights. I have deal it by creating equal mean (Standard Mean) of the data. However Weights generally leads to create the standard error.

• The income values present need adjustment which was clearly mention the Data Dictionary. It clearly shows that the income value should be adjusted with ‘ADJHSG’ or with the ‘ADJINC’.

Did you produce any tables or plots that you thought would reveal interesting trends but didn’t?

• 1.In order to give the states wise family members plotting , I was trying to create a Circular Bar Plot which gives a good view of graph and clearly depicts the results in fancy way but I was not able to make the create the actuall angles which helps me to divide the circular plot into cluster.

• 2.I was also trying to make the dense Heat map which shows the relation between the Value of the Property and the Amenities Present in the House but due to the factor plotting I was not able to create in that heat map , the darker section shows that the house the maximum amenities holds the maximum value in terms of values of the price.

What’s the analysis that you finally settled on? What relationships do you investigate in the final analysis?

• In the findings section I have analysed the major relationship between the economic variables/columns present in the data which actually tells about the economic circumstances of a particular demographics. I have also compared the state wise section which differentiate the coherence and distinctiveness of a state.

• Along with that I have also test the actual fact which is a TRUE FACT accroding the statistics on internet and confirm it whether that is actually true or not.

• I have also predict the values using Predictive Analysis - Linear Regression, Chi-Square Test and I have also perform the prediction of houses value Using Support Vector Machine (introduce the CARET library).

• I have also introduce the Dynamic Graphs using plotly where we can filter the graph on the basis of condition and also check the value of each section along wth its plotting attribute.

3. Findings

Following are the findings I come up with (It consist of the Tabular Summaires , Graphical Summeries , All the predictive outputs , Statistical Significance) :

## # A tibble: 51 x 2
##       ST Avg_Value
##    <int>     <dbl>
##  1     1   165098.
##  2     2   236745.
##  3     4   241961.
##  4     5   142313.
##  5     6   584500.
##  6     8   342265.
##  7     9   397463.
##  8    10   286793.
##  9    11   666619.
## 10    12   263859.
## # … with 41 more rows

##        WIF HINCP GRNTP RNTP FINCP
## WIF   1.00  0.33  0.22 0.20  0.35
## HINCP 0.33  1.00  0.49 0.48  0.98
## GRNTP 0.22  0.49  1.00 0.98  0.48
## RNTP  0.20  0.48  0.98 1.00  0.48
## FINCP 0.35  0.98  0.48 0.48  1.00

##        WIF HINCP GRNTP RNTP FINCP
## WIF   1.00  0.33  0.22 0.20  0.35
## HINCP 0.33  1.00  0.49 0.48  0.98
## GRNTP 0.22  0.49  1.00 0.98  0.48
## RNTP  0.20  0.48  0.98 1.00  0.48
## FINCP 0.35  0.98  0.48 0.48  1.00
## 
## n= 893525 
## 
## 
## P
##       WIF HINCP GRNTP RNTP FINCP
## WIF        0     0     0    0   
## HINCP  0         0     0    0   
## GRNTP  0   0           0    0   
## RNTP   0   0     0          0   
## FINCP  0   0     0     0

## 
##  F test to compare two variances
## 
## data:  predict.data$RMSP and predict.data$VALP
## F = 0.000000000036489, num df = 4347579, denom df = 4347579,
## p-value < 0.00000000000000022
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
##  0.00000000003644095 0.00000000003653797
## sample estimates:
##  ratio of variances 
## 0.00000000003648941

## [1] 0.2500915

## [1] 1.367129

## [1] 6.586461

## 
## Call:
## lm(formula = valueofhouse ~ RMSP, data = predict.data.test)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -389784 -129011  -31586   50375  632753 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   422720      93541   4.519  0.00013 ***
## RMSP          -18546       5768  -3.215  0.00358 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 239500 on 25 degrees of freedom
## Multiple R-squared:  0.2925, Adjusted R-squared:  0.2642 
## F-statistic: 10.34 on 1 and 25 DF,  p-value: 0.003579

## [1] 134.3137

## [1] -0.002725119

## [1] FALSE

##      FINCP             VALP        
##  Min.   :     0   Min.   :   1700  
##  1st Qu.: 43600   1st Qu.:  70000  
##  Median : 62780   Median : 122500  
##  Mean   : 84840   Mean   : 173947  
##  3rd Qu.:106138   3rd Qu.: 215500  
##  Max.   :444000   Max.   :1843000

## Support Vector Machines with Linear Kernel 
## 
## 72 samples
##  1 predictor
## 42 classes: '1700', '12000', '13000', '20000', '25000', '30000', '35000', '40000', '50000', '60000', '65000', '70000', '75000', '80000', '84000', '90000', '96000', '100000', '103000', '115000', '120000', '125000', '130000', '135000', '140000', '150000', '160000', '180000', '190000', '200000', '215000', '217000', '220000', '225000', '250000', '300000', '350000', '375000', '400000', '500000', '750000', '829000' 
## 
## Pre-processing: centered (1), scaled (1) 
## Resampling: Cross-Validated (10 fold, repeated 3 times) 
## Summary of sample sizes: 66, 63, 67, 66, 61, 64, ... 
## Resampling results:
## 
##   Accuracy    Kappa     
##   0.03821225  0.02033985
## 
## Tuning parameter 'C' was held constant at a value of 1

##  [1] 150000 125000 125000 125000 125000 250000 125000 125000 40000  125000
## [11] 40000  125000 40000  125000 125000 125000 150000 125000 125000 125000
## [21] 150000 40000  125000 125000 125000 125000 250000 40000 
## 42 Levels: 1700 12000 13000 20000 25000 30000 35000 40000 50000 ... 829000

## 
##  Pearson's Chi-squared test
## 
## data:  tbltest
## X-squared = 912600, df = 18, p-value < 0.00000000000000022

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##       4     670     940    1071    1336    5022

## 
##  One-sample Kolmogorov-Smirnov test
## 
## data:  gross.rent$GRNTP
## D = 0.11121, p-value < 0.00000000000000022
## alternative hypothesis: two-sided

## 
##  One Sample t-test
## 
## data:  gross.rent$GRNTP
## t = 126.56, df = 1725631, p-value < 0.00000000000000022
## alternative hypothesis: true mean is not equal to 1012
## 95 percent confidence interval:
##  1070.051 1071.877
## sample estimates:
## mean of x 
##  1070.964

4. Discussion

After effectuating and completing the various noteworthy and significant analysis , I have come up with various outcomes and result which clearly tells about the housing pattern in the United States of America and distinguish between each and every variable in terms of maintaning the economy. Through this scrutiny , I got to know more about the demographical information of the United States of America.

Following are the underlying conceptions which yields out of this analysis:-

1. From the Statewise Analysis , I confirm that the New York is the state with maximum land value - Initially I only heard about that but now I have the concrete evidence to prove the same fact. New York followed by California is the most expensive state in terms of land value and that is point of noting for some one who is working on the economic sector of the United States.

2. From the analysis I realize that still many people in United States are actually craving for good internet speed which is point worth noting for tele-communication sector. Also the Natural Resources consumption varies from states to states which is not much influential , though we can have a rough idea about that now.

3. Since this is a housing data , there is a pure chance of calculating the house related tax. From this analysis I come up with a conclusion that the state with the maximum land value is the only state which costs higher house tax. Although I was bit shocked to know that Louisiana costs less in terms of Housing Tax (being the 3rd most populated state with 4.66 Million Americans)

4. Using the predictive analysis I confirmed that each economic variable in the data is quite important to one other (finding the correlation). The impact on one can be mirror impact on another which is was not quite obvious. In the predictive section I have predicted that whether the Room Numbers actually impact the House Value or not. Confirming the relation I can say that this matters in many different scenarios. Getting an extra room in your house in the United States of America can losen up your pocket. I have also find out that the household income wont decide the Area of the house , the family is living in.

5. From the analysis I found that Americans dont care much about their houses and invest almost 1/10th of their salary on house insurance. Infact I checked that on major 5 cities including New York but did not get much fruitful results which makes New York exceptional.

6. I also got to know that the Year of Construction also decided the value of the House which is not pretty much abvious in every country.The houses constructed after the year 2015 has the maximum value in a decade. Also , the United States consist of lot of Spaniards. Spanish speaking people are the second largest housing in United States after English Speakers.

7. After analysis , I was so fascinated to know that many houses in United States of America has zero worker in a home. Though there are some chance of discripency here because survival in country like United States without earning is quite difficult (Almost Impossible).

8. Using the Chi Square test , I found that houses with most family also owns the most number of cars in a home which tells that people of United States are concerned about the transportation but not about living stakes.

Are the models you fit believable?

I belive that the models I fit and used are the right models to work upon but in my case the only concern is about traning them in a recursive way so that they yield accpetable accuracy (which I some time lacks). I can also concern here about the machine because most of the model training is done on cloud based technology.

How much confidence do you have in your analysis? Do you believe your conclusions? Are you confident enough in your analysis and findings to present them to policy makers?

I have around 80%-85% confidence and sure about the conclusion I have constructed. Also there is always a room of Improvement everywhere. The conclusion I have constructed is completely helpful for Policy Simulation and Creation in a Country. I completely believe in the effort I have put on and also believes that this is quite helpful for a viewer. I am confident about my outputs and results here. Although I need work more upon the data plotting part.

5. Overall Code and Appendix

Code-1 : Reading the data here inside this chunk

# reading the data set only

library(data.table)
library(pander)
library(kableExtra)
library(skimr)


# start_time <- Sys.time()
# mainData <- fread("MainData.csv")
# end_time <- Sys.time()


#end_time - start_time
options(scipen=10000)

start_time <- Sys.time()

# selecting the column on which I have to work upon
cols <- c("ST", "VALP","LAPTOP","HISPEED","FULP","GASP","TYPE","TAXP","WIF","HINCP","GRNTP","RNTP","FINCP","RMSP","NP","ADJINC","ACR","INSP","FTAXP","FHINCP","FINSP","YBL","MV","HHL","VEH")

# using the fread method of the data.table package to read the huge data in less time.
mainData <- fread("MainData.csv",select = cols)
end_time <- Sys.time()

# calculating the time of reading here
end_time - start_time


# getting the fancy summary of data using pander package
pander(summary(mainData))

# getting the summary plus histogram of data using the skim package
skim(mainData)

Code-2 : Getting the insight of data here

library(kableExtra)
library(dplyr)
library(magrittr)
library(DT)
options(DT.options = list(pageLength = 30))


top10.mainData <- mainData[1:10]
top10.mainData %>%
    datatable(options = list(dom = "t", ordering = FALSE), 
              rownames = FALSE,
              width = 30) %>%
    formatStyle(c("ST", "VALP","LAPTOP","HISPEED","FULP","GASP","TYPE","TAXP","WIF","HINCP","GRNTP","RNTP","FINCP","RMSP","NP","ADJINC","ACR","INSP","FTAXP","FHINCP","FINSP","YBL","MV","HHL","VEH"), backgroundColor = styleEqual(NA, "skyblue"))

Code-3 : Checking the unwanted and null values present in the data

library(dplyr)
library(tidyverse)
library(ggdark)
library(ggplot2)



missing.values <- mainData %>%
    gather(key = "key", value = "val") %>%
    mutate(is.missing = is.na(val)) %>%
    group_by(key, is.missing) %>%
    summarise(num.missing = n()) %>%
    filter(is.missing==T) %>%
    select(-is.missing) 

missing.values$id <- seq(1,21)


label_data <- missing.values
number_of_bar <- nrow(label_data)
angle <-  90 - 360 * (label_data$id) /number_of_bar
label_data$hjust<-ifelse( angle < -90, 1, 0)
label_data$angle<-ifelse(angle < -90, angle+180, angle)

ggplot(missing.values, aes(x=as.factor(id), y=num.missing)) +       # Note that id is a factor. If x is numeric, there is some space between the first bar
  
  # This add the bars with a blue color
  geom_bar(stat="identity", fill=alpha("yellow", 1)) +
  
  # Limits of the plot = very important. The negative value controls the size of the inner circle, the positive one is useful to add size over each bar
  ylim(-1,120) +
  
  # Custom the theme: no axis title and no cartesian grid
  theme_minimal() +
  theme(
    axis.text = element_blank(),
    axis.title = element_blank(),
    panel.grid = element_blank(),
    plot.margin = unit(rep(-1,4), "cm")      # Adjust the margin to make in sort labels are not truncated!
  ) +
  
  # This makes the coordinate polar instead of cartesian.
  coord_polar(start = 0) +
  
  # Add the labels, using the label_data dataframe that we have created before
  geom_text(data=label_data, aes(x=id, y=num.missing+10, label=key, hjust=hjust), color="White", fontface="bold",alpha=0.6, size=2.5, angle= label_data$angle, inherit.aes = FALSE ) + scale_y_continuous(limits=c(0,100) + scale_x_continuous(limits=c(0,100))) + dark_theme_void() + labs(title = "Circular Bar Plot of missing values")

Code-4 : Calculating the Null values and creating the insight

library(dplyr)
library(ggplot2)
library(plotly)
library(tidyverse)
library(ggdark)


missing.values.percentage <- mainData %>%
  gather(key = "key", value = "val") %>%
  mutate(isna = is.na(val)) %>%
  group_by(key) %>%
  mutate(total = n()) %>%
  group_by(key, total, isna) %>%
  summarise(num.isna = n()) %>%
  mutate(pct = num.isna / total * 100)




levels <-
    (missing.values.percentage  %>% filter(isna == T) %>% arrange(desc(pct)))$key

percentage.plot <- missing.values.percentage %>%
      ggplot() +
        geom_bar(aes(x = reorder(key, desc(pct)), 
                     y = pct, fill=isna , width=.8), 
                 stat = 'identity', alpha=0.35 , position = position_dodge() , colour="black") +
      scale_x_discrete(limits = levels) +
      scale_fill_manual(name = "", 
                        values = c('purple3', 'gold1'), labels = c("Present", "Missing")) +
      coord_flip() +
      labs(title = "Percentage of missing values", x =
             'Variable', y = "% of missing values")


percentage.ploting <- ggplotly(percentage.plot)
percentage.ploting

Code-5 : Calculating and plotting the average property value in USA by states

library(dplyr)
library(ggplot2)
library(ggdark)

valuebystate <- mainData %>% select(ST , VALP)
plotdata <- valuebystate %>% group_by(ST) %>% filter(!any(is.na(ST))) %>% summarise(Avg_Value = mean(VALP,na.rm = TRUE))

statewise.landValue <- valuebystate %>% group_by(ST) %>% filter(!any(is.na(ST))) %>% summarise(Avg_Value = mean(VALP,na.rm = TRUE))
statewise.landValue
tb <- (plotdata$ST)
tbNorm <- (plotdata$Avg_Value)

require ("ggplot2")
require ("choroplethr")
require ("choroplethrMaps")


states <- data.frame ( c (1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20,
21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51,
53, 54, 55, 56, 72),
    c("Alabama", "Alaska", "Arizona", "Arkansas", "California", "Colorado", "Connecticut",
    "Delaware", "District of Columbia", "Florida", "Georgia", "Hawaii", "Idaho", "Illinois",
    "Indiana", "Iowa", "Kansas", "Kentucky", "Louisiana", "Maine", "Maryland", "Massachusetts",
    "Michigan", "Minnesota", "Mississippi", "Missouri", "Montana", "Nebraska", "Nevada",
    "New Hampshire", "New Jersey", "New Mexico", "New York", "North Carolina", "North Dakota",
    "Ohio", "Oklahoma", "Oregon", "Pennsylvania", "Rhode Island",
    "South Carolina", "South Dakota", "Tennessee", "Texas", "Utah", "Vermont", "Virginia",
    "Washington", "West Virginia", "Wisconsin", "Wyoming", "Puerto Rico"))
    
names (states) = c ("Index", "State")

# takes a table across states as input and prints it on a map
# uses the "choroplethr" package
plot.map <- function (tb, title = "", legend = "") {
 
  i <- match (names (tb), as.character(states$Index))
  states <- data.frame (tb, tolower(states$State[i]))
  states <- states [,c (3,2)]
  
  # this is required by choroplethr
  names (states) <- c ("region", "value")
  states$region <- as.character (states$region)
  
  # this is identical to state_choroplethr except that the labels are being removed
  c = StateChoropleth$new(states)
  c$set_num_colors(7)
  c$title  = title
  c$legend = legend
  c$set_zoom(NULL)
  c$show_labels = FALSE
  c$render()
}

avg.val.data <- (statewise.landValue$Avg_Value)
state.data <- (plotdata$ST)
avg.data <- state.data / avg.val.data
geographical.rep.table <- as.table(setNames(tbNorm,tb))
plot.map (geographical.rep.table, "Average Property Value By States in US", "Cluster of Land Value") + theme(plot.title = element_text(hjust = 0.5)) + dark_theme_linedraw() + theme(legend.background = element_rect(fill="black",
                                  size=0.5, linetype="solid", 
                                  colour ="white"))

Code-6 : Calculating and plotting the Laptop users with High speed interent availablity

user.laptop.hispeed <- mainData %>% filter(LAPTOP != 'b' , LAPTOP ==1 , HISPEED !='NA' ) %>% select(LAPTOP , HISPEED)
user.laptop.hispeed.pie <- user.laptop.hispeed %>% group_by(LAPTOP,HISPEED) %>% summarise(COUNT = n()) %>% mutate(lab.ypos = cumsum(COUNT) - 0.5*COUNT)
user.laptop.hispeed.pie$HISPEED <- factor(user.laptop.hispeed.pie$HISPEED , labels = c("Yes" , "No"))


ggplot(user.laptop.hispeed.pie, aes(x = "", y = COUNT , fill = HISPEED)) +
  geom_bar(width = 1, stat = "identity", color = "black") +
  coord_polar("y", start = 0) + 
  theme_void() + geom_text(aes(label = paste0(COUNT, " (", scales::percent(COUNT / sum(COUNT)),")")),
position = position_stack(vjust = 0.8) , check_overlap = T , size = 3.5) + labs(title = "Desktop Users With HighSpeed Internet Service" , fill = "Hispeed Connectivity") + theme(legend.text = element_text(face = "italic", colour="steelblue4",family = "Helvetica"),legend.title = element_text(colour = "steelblue",  face = "bold.italic", family = "Helvetica")) + theme(
  legend.box.background = element_rect(),
  legend.box.margin = margin(6, 6, 6, 6)
)

Code-7 : Calculating and plotting the natural resources consumption in united states

library(tidyverse)

# Natural Resources Consumption by States
FuelCost <- mainData %>% select(ST , FULP , GASP) %>% filter(FULP > 2 , GASP >2)
FuelCostPlot <- FuelCost %>% group_by(ST) %>% summarise(Avg_Fuel_Usage = mean(FULP , na.rm = TRUE) , Avg_Gas_Usage = mean(GASP , na.rm = TRUE))


FuelCostPlot$ST <- factor(FuelCostPlot$ST)
levels(FuelCostPlot$ST) <- c("Alabama", "Alaska", "Arizona", "Arkansas", "California", "Colorado", "Connecticut",
    "Delaware", "District of Columbia", "Florida", "Georgia", "Hawaii", "Idaho", "Illinois",
    "Indiana", "Iowa", "Kansas", "Kentucky", "Louisiana", "Maine", "Maryland", "Massachusetts",
    "Michigan", "Minnesota", "Mississippi", "Missouri", "Montana", "Nebraska", "Nevada",
    "New Hampshire", "New Jersey", "New Mexico", "New York", "North Carolina", "North Dakota",
    "Ohio", "Oklahoma", "Oregon", "Pennsylvania", "Rhode Island",
    "South Carolina", "South Dakota", "Tennessee", "Texas", "Utah", "Vermont", "Virginia",
    "Washington", "West Virginia", "Wisconsin", "Wyoming", "Puerto Rico")

state.wise.consumption.data <- FuelCostPlot %>%
  gather("Stat", "Value", -ST)

ggplot(state.wise.consumption.data, aes(x = ST, y = Value, fill = Stat)) +
  geom_col(position = "dodge") + coord_flip() +scale_fill_manual(values=c("#D823DE", "#23DEA0"))  + theme(legend.box.background = element_rect(),legend.box.margin = margin(6, 6, 6, 6)) + labs(title = "Natural Resources Consumption by States") + theme(axis.title.x = element_blank(),axis.text.y=element_text(size=rel(0.8))) + dark_theme_linedraw() + xlab("State Names") +ylab("Consumption Values") + theme(legend.background = element_rect(fill="black",
                                  size=0.5, linetype="solid", 
                                  colour ="white"))

Code-8 : Calculating and plotting the average house tax by states in the United States

library(mapproj)
library(ggplot2)
library(ggdark)


states_data_map <- map_data("state")

Houses.only <- mainData %>% filter(TYPE == 1 , FTAXP == 1) %>% select(ST , TAXP)


avg.tax <- Houses.only %>% group_by(ST) %>% summarise(avg.mean.tax = mean(TAXP , na.rm = T))

ST<-c(1,2,4,5,6,8,9,10,11,12,13,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,44,45,46,47,48,49,50,51,53,54,55,56,72)
region<-c('alabama','alaska','arizona','arkansas','california','colorado','connecticut','delaware','district of columbia','florida','georgia','hawaii','idaho','illinois','indiana','iowa','kansas','kentucky','louisiana','maine','maryland','massachusetts','michigan','minnesota','mississippi','missouri','montana','nebraska','nevada','new hampshire','new jersey','new mexico','new york','north carolina','north dakota','ohio','oklahoma','oregon','pennsylvania','rhode island','south carolina','south dakota','tennessee','texas','utah','vermont','virginia','washington','west virginia','wisconsin','wyoming','puerto rico')

states.data <- data.frame(ST,region)

states.data$ST <- as.factor(as.character(states.data$ST))
avg.tax$ST <- as.factor(as.character(avg.tax$ST))

common.data <-   merge(avg.tax,states.data,by="ST")
mapcreation  <-  merge(states_data_map, common.data, by="region")

centroids <- data.frame(region=tolower(state.name), long=state.center$x, lat=state.center$y)
centroids$abb<-state.abb[match(centroids$region,tolower(state.name))]


ggplot(mapcreation, aes(x = long, y = lat, group = group, fill = avg.mean.tax)) + 
coord_map("gilbert") +
theme_light() +
scale_fill_continuous(low="Green",high="red",limits=c(min(mapcreation$avg.mean.tax), max(mapcreation$avg.mean.tax))) +
labs(title = "Average House Tax (Property) Figures - By States",fill="Average\nProperty Tax\n Costs($)") + geom_polygon(colour = "black") + theme(strip.background = element_blank(), strip.text.x = element_blank(), axis.text.x = element_blank(), axis.text.y = element_blank(), axis.ticks  = element_blank(), axis.line   = element_blank(), panel.border= element_blank(), panel.grid  = element_blank(), legend.position = "right") + xlab("") + ylab("") + with(centroids,  ggplot2::annotate(geom="text", x = long, y=lat, label = abb, 
size = 3,color="black",family="Times")) + theme(legend.box.background = element_rect(),legend.box.margin = margin(6, 6, 6, 6)) + theme(plot.title = element_text(hjust = 0.5)) + theme(plot.title = element_text(hjust = 0.5)) + dark_theme_linedraw() + theme(legend.background = element_rect(fill="black",
                                  size=0.5, linetype="solid", 
                                  colour ="white"))

Code-9 : Checking and plotting the Economic entities in the United States

# Coorelation between all economic aspect columns which proves the basic income and expenditure in a household.
require(ggpubr)
require(tidyverse)
require(Hmisc)
require(corrplot)
library(ellipse)
library(RColorBrewer)
corr.components <- mainData %>% select(WIF , HINCP , GRNTP , RNTP , FINCP )
corr.components <- na.omit(corr.components)


round(cor(corr.components), 2)
rcorr(as.matrix(corr.components))

M<-cor(corr.components)
corrplot(M, method = "ellipse",col=brewer.pal(n=8, name="PuOr"))
corrplot(M, method = "number",col=brewer.pal(n=8, name="PuOr"))

Code-10 : Predict the Value of the House by the Number of Rooms Available

library(moments)
library(dplyr)
library(ggplot2)
library(knitr)
library(kableExtra)
library(broom)
library(ggdark)


# Extracting the valueable columns
predict.data <- mainData %>% filter(RMSP != 'NA' , VALP != 'NA' ) %>% select(RMSP , VALP)
predict.data <- na.omit(predict.data)

# testing the variance of dataset
var.test(predict.data$RMSP,predict.data$VALP)


# checking the correlation between the variables after the filtering.
cor(predict.data$VALP , predict.data$RMSP)


# Checking the normality

#checking the skewness of RMSP column
skewness(predict.data$RMSP)

#checking the skewness of VALP column
skewness(predict.data$VALP)

ggplot(predict.data, aes(x=RMSP)) + 
 geom_histogram(aes(y=..density..), colour="darkblue", fill="lightblue")+
 geom_density(alpha=.2, fill="#FF6666" , color = 'black') + labs(title="Histogram for Room Number") +
  labs(x="Number of Rooms", y="Desnity Values") + theme(plot.title = element_text(hjust = 0.5))  + geom_vline(aes(xintercept=mean(RMSP)),
            color="blue", linetype="dashed", size=1)


ggplot(predict.data, aes(x=VALP)) + 
 geom_histogram(aes(y=..density..), colour="darkblue", fill="red")+
 geom_density(alpha=.2, fill="#FF6666",color = 'black') + labs(title="Histogram for Value of House") +
  labs(x="Value of House", y="Desnity Values") + theme(plot.title = element_text(hjust = 0.5))  + geom_vline(aes(xintercept=mean(RMSP)),
            color="red", linetype="dashed", size=1)

# checking the relation by visualization
ggplot(predict.data , aes(x = VALP , y = RMSP)) + geom_point(color = "black") + stat_smooth()

predict.data.test <- predict.data %>% group_by(RMSP) %>% dplyr::summarise(valueofhouse = n())

lm.method.predict <- lm(VALP ~ RMSP, data =
             predict.data)

lm.method.predict.test <- lm(valueofhouse ~ RMSP, data =
             predict.data.test)


summary(lm.method.predict.test)

lm.method.predict.summary <- summary(lm.method.predict.test)

kable(lm.method.predict.summary$coefficients) %>%
  kable_styling("striped", full_width = F) %>%
  row_spec(0:2, bold = T, color = "black", background = "skyblue")

sigma(lm.method.predict)*100/mean(predict.data$VALP)

qqnorm(lm.method.predict$residuals, col = "yellow")
qqline(lm.method.predict$residuals, col = "blue")

lm.method.predict.test.metrics <- augment(lm.method.predict.test)

ggplot(lm.method.predict.test.metrics, aes(RMSP, valueofhouse)) +
  geom_point() +
  stat_smooth(method = lm, se = FALSE) +
  geom_segment(aes(xend = RMSP, yend = .fitted), color = "red", size = 0.3) + annotate("rect", xmin=c(3.5,15), xmax=c(9.5,26), ymin=c(250000,0) , ymax=c(750000,150000), alpha=0.2, color="white", fill="blue") + dark_theme_linedraw() + annotate("text", x=11, y=820000, label= "Maximum Residual Value \n (R.S.S value very high)") + annotate("text", x=22, y=220000, label= "Minimum Residual Value \n (R.S.S value Accurate)") + xlab("Room Space") + ylab("Value of the House")

Code-11 : Does House Income depicts the standard of Living in the United States

require(ggpubr)
# Correlation between the Income and the House size

income.house.data <- mainData %>% select(HINCP , NP , ADJINC , ACR) %>% filter(ACR != 'NA')
income.house.data <- na.omit(income.house.data)

income.house.data$AdjustedHousehold_Inc<-income.house.data$HINCP*unique(income.house.data$ADJINC/1E06,incomparables = FALSE)

# removing the outliers

outliers<-boxplot(income.house.data$AdjustedHousehold_Inc,plot=F)$out

# finding the corelation

plot(income.house.data$ACR,income.house.data$AdjustedHousehold_Inc,col="blue")
income.house.data.df<- income.house.data[,c("AdjustedHousehold_Inc","ACR")]
income.house.data.df1<- income.house.data.df[complete.cases(income.house.data.df),]
cor(income.house.data.df1$AdjustedHousehold_Inc,as.numeric(income.house.data.df1$ACR))

ggscatter(income.house.data.df1, x = "AdjustedHousehold_Inc", y = "ACR",
   add = "reg.line",
   add.params = list(color = "blue", fill = "lightgray"),
   conf.int = TRUE
   ) + stat_cor(method = "pearson", label.x = 3, label.y = 30)

Code-12 : How much Americans Care About Their Houses.

library(ggdark)
library(viridis)

#how much americans care about their homes

house.insurance.care <- mainData %>% select(TYPE , FINSP , FHINCP , HINCP , INSP , ST)


# gettng the type of unit = 1 , Fire, hazard, flood insurance  = 1 and household income
house.insurance.cols <-  subset(house.insurance.care,TYPE==1 & FINSP==1 & FHINCP==1) 
house.data<- house.insurance.cols[,c('ST','INSP','HINCP')]

unique.states <- unique(house.data$ST)

insurance.data <- as.data.frame(tapply(house.data$INSP,house.data$ST,mean))
colnames(insurance.data) <- c("Avg")
dataframe.insu <- as.data.frame(cbind(ST=unique.states,Avg=insurance.data$Avg))
dataframe.insu$flag <- c("Insurance")
income.data = as.data.frame(tapply(house.data$HINCP,house.data$ST,mean))
colnames(income.data) <- c("Avg")
dataframe.inc <- as.data.frame(cbind(ST=unique.states,Avg=income.data$Avg))
dataframe.inc$flag <-  c("Income")
dataframe.insu$ST=as.factor(as.character(dataframe.insu$ST))
dataframe.inc$ST=as.factor(as.character(dataframe.inc$ST))

insurance.house.df <- rbind(dataframe.insu,dataframe.inc)
ST<-c(1,2,4,5,6,8,9,10,11,12,13,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,44,45,46,47,48,49,50,51,53,54,55,56,72)
region<-c('alabama','alaska','arizona','arkansas','california','colorado','connecticut','delaware','district of columbia','florida','georgia','hawaii','idaho','illinois','indiana','iowa','kansas','kentucky','louisiana','maine','maryland','massachusetts','michigan','minnesota','mississippi','missouri','montana','nebraska','nevada','new hampshire','new jersey','new mexico','new york','north carolina','north dakota','ohio','oklahoma','oregon','pennsylvania','rhode island','south carolina','south dakota','tennessee','texas','utah','vermont','virginia','washington','west virginia','wisconsin','wyoming','puerto rico')
housing.states <- data.frame(ST,region)
housing.states$ST <- as.factor(as.character(housing.states$ST))
insurance.house.df$ST=as.factor(as.character(insurance.house.df$ST))

all.housing.insu <- merge(insurance.house.df,housing.states,by="ST")
all.housing.insu.top5 = subset(all.housing.insu, ST ==6 | ST == 11 | ST == 25 |ST == 36 |ST == 53 )

ggplot(data=all.housing.insu, aes(x=region, y=round(Avg), fill=flag)) +
    geom_bar(stat="identity", position=position_dodge())+ coord_flip() +
theme(axis.title.x = element_blank(),axis.text.y=element_text(size=rel(0.8)))  + scale_fill_manual(values=c("#23DEDB", "#DEDE23")) +theme(legend.box.background = element_rect(),legend.box.margin = margin(6, 6, 6, 6)) + theme(plot.title = element_text(hjust = 0.5)) + dark_theme_linedraw() + theme(legend.background = element_rect(fill="black",
                                  size=0.5, linetype="solid", 
                                  colour ="white"))

Code-13 : Value of Houses based On their Construction Year

library(ggplot2)
library(dplyr)
library(ggdark)

options(scipen=10000)

yearbuild.propertyval <- mainData %>% select(YBL , VALP)
yearbuild.propertyval <- na.omit(yearbuild.propertyval)
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 01 , "1939 or earlier")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 02 , "1940 to 1949")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 03 , "1950 to 1959")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 04 , "1960 to 1969")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 05 , "1970 to 1979")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 06 , "1980 to 1989")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 07 , "1990 to 1999")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 08 , "2000 to 2004")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 09 , "2005")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 10 , "2006")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 11 , "2007")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 12 , "2008")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 13 , "2009")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 14 , "2010")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 15 , "2011")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 16 , "2012")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 17 , "2013")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 18 , "2014")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 19 , "2015")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 20 , "2016")
yearbuild.propertyval$YBL <- replace(yearbuild.propertyval$YBL, yearbuild.propertyval$YBL == 21 , "2017")

avg.price.yearwise <- yearbuild.propertyval %>% group_by(YBL) %>% summarise(mean.value = mean(VALP , na.rm = T))

ggplot(avg.price.yearwise, aes(x=`YBL`, y=mean.value , label = "")) + 
  geom_point(stat='identity', fill="black", size=6 , color="Red" , alpha = 0.6)  +
  geom_segment(aes(y = 0, 
                   x = `YBL`, 
                   yend = mean.value, 
                   xend = `YBL`), 
               color = "white") +
  geom_text(color="white", size=3) +
  labs(title="Value of House Based on Construction Year", 
       subtitle="Does Construction Year Matters ?") + coord_flip() + dark_theme_linedraw() + xlab("Year of Build") + ylab("Average Price of House")

Code-14 : Linguistic Demographics versus Year of living

library(ggplot2)
library(dplyr)
library(ggdark)

move.data <- mainData %>% select(MV,HHL) %>% filter(MV != "NA" , HHL != "NA") %>% na.omit(move.data)
move.data$MV <- replace(move.data$MV, move.data$MV == 1 , "12 months or less")
move.data$MV <- replace(move.data$MV, move.data$MV == 2 , "13 to 23 months")
move.data$MV <- replace(move.data$MV, move.data$MV == 3 , "2 to 4 years")
move.data$MV <- replace(move.data$MV, move.data$MV == 4 , "5 to 9 years")
move.data$MV <- replace(move.data$MV, move.data$MV == 5 , "10 to 19 years")
move.data$MV <- replace(move.data$MV, move.data$MV == 6 , "20 to 29 years")
move.data$MV <- replace(move.data$MV, move.data$MV == 7 , "30 years or more")


move.data$HHL <- replace(move.data$HHL, move.data$HHL == 1 , "English only")
move.data$HHL <- replace(move.data$HHL, move.data$HHL == 2 , "Spanish")
move.data$HHL <- replace(move.data$HHL, move.data$HHL == 3 , "Other Indo-European languages")
move.data$HHL <- replace(move.data$HHL, move.data$HHL == 4 , "Asian and Pacific Island languages")
move.data$HHL <- replace(move.data$HHL, move.data$HHL == 5 , "Other language")

move.data <- move.data %>% group_by(MV, HHL) %>% summarise(count.values = n())

ggplot() + geom_point(data = move.data, aes(x = MV, y = count.values, size = 5, color = HHL, shape = HHL)) + coord_flip() +dark_theme_linedraw()  + xlab("Living Here Since..") + ylab("Values") + labs(title = "Migrate Period Along with Linguistic Demographics")

Code-15 : Predicting the value of the house by family Income Using Support Vector Machine

library(caret)
library(dplyr)
library(kernlab)

# getting the data
f.income.house.size <- mainData %>% select(FINCP,VALP) %>% filter(FINCP != "NA" , VALP != "NA" ) %>% na.omit(f.income.house.size)
# selecting the minimum data because of machine limit
f.income.house.size <- head(f.income.house.size,100)


# partitioning the data
intrain <- createDataPartition(y = f.income.house.size$VALP, p= 0.7, list = FALSE)
training <- f.income.house.size[intrain,] 
testing <- f.income.house.size[-intrain,]
training[["VALP"]] = factor(training[["VALP"]])

# anyNA() method, which checks for any null values
anyNA(f.income.house.size)

# summary of the data
summary(f.income.house.size)

# assigning the train control method here. Control all the computational overheads so that we can use the train() function provided by the caret package
trctrl <- trainControl(method = "repeatedcv", number = 10, repeats = 3)


# training the model usin
svm_Linear <- train(VALP ~ FINCP, data = training, method = "svmLinear",
trControl=trctrl,
preProcess = c("center", "scale"),
tuneLength = 10)

# predicting the model
svm_Linear

# prediciting the respective values
test_pred <- predict(svm_Linear, newdata = testing)
test_pred

Code-16 : State Wise - Number of Workers Present in a House (Dynamic Graph)

library(dplyr)
library(ggplot2)
library(viridis)
library(hrbrthemes)
library(ggdark)
library(plotly)



state.workers.home <- mainData %>% select(ST, WIF) %>% filter(WIF != "NA") %>% na.omit(state.workers.home)
state.workers.home <- state.workers.home %>% dplyr::group_by(ST,WIF) %>% dplyr::summarise(count = n())

# Changing the section here.
state.workers.home$WIF <- replace(state.workers.home$WIF, state.workers.home$WIF == 0 , "Zero Worker")
state.workers.home$WIF <- replace(state.workers.home$WIF, state.workers.home$WIF == 1 , "One Worker")
state.workers.home$WIF <- replace(state.workers.home$WIF, state.workers.home$WIF == 2 , "Two Worker")
state.workers.home$WIF <- replace(state.workers.home$WIF, state.workers.home$WIF == 3 , "Three Worker")

# Changing the state names here.

state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 1,"alabama")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 2,"alaska")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 4,"arizona")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 5,"arkansas")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 6,"california")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 8,"colorado")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 9,"connecticut")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 10,"delaware")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 11,"district of columbia")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 12,"florida")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 13,"georgia")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 15,"hawaii")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 16,"idaho")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 17,"illinois")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 18,"indiana")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 19,"iowa")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 20,"kansas")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 21,"kentucky")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 22,'louisiana')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 23,"maine")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 24,"maryland")
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 25,'massachusetts')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 26,'michigan')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 27,'minnesota')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 28,'mississippi')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 29,'missouri')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 30,'montana')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 31,'nebraska')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 32,'nevada')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 33,'new hampshire')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 34,'new jersey')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 35,'new mexico')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 36,'new york')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 37,'north carolina')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 38,'north dakota')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 39,'ohio')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 40,'oklahoma')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 41,'oregon')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 42,'pennsylvania')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 44,'rhode island')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 45,'south carolina')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 46,'south dakota')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 47,'tennessee')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 48,'texas')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 49,'utah')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 50,'vermont')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 51,'virginia')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 53,'washington')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 54,'west virginia')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 55,'wisconsin')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 56,'wyoming')
state.workers.home$ST <- replace(state.workers.home$ST, state.workers.home$ST == 72,'puerto rico')


p <- ggplot(state.workers.home, aes(fill=WIF, y=count, x=ST)) + 
    geom_bar(position="stack", stat="identity") +
    scale_fill_viridis(discrete = T) +
    ggtitle("Statewise - Number of Workers Present in Home") +
    theme_ipsum() +
    xlab("State Names") + theme(axis.text.x = element_text(angle=90, hjust=1, size = 7))

ploting <- ggplotly(p)
ploting

Code-17 : Finding the Relationship between Number of Workers in a House and the Vehciles they Own

# relationship bw workers in the family and monthly rent

library(dplyr)
library(gplots)
library(ggdark)

worker.vehciles <- mainData %>% select(WIF, VEH) %>% filter(WIF != "NA" , VEH != 'NA') %>% na.omit(worker.vehciles)
tbltest <- table(worker.vehciles$WIF,worker.vehciles$VEH)

# performing the chi square test 
chisq.test(tbltest)

balloonplot(t(tbltest), main ="Workers in House - Vehciles They Own", xlab ="Vehicles", ylab="No of Workers",
            label = FALSE, show.margins = FALSE)

#create density curve
curve(dchisq(x, df = 18), from = 0, to = 40,
main = 'Chi-Square Distribution (df = 18)',
ylab = 'Density',
lwd = 2)

Code-18 : Compare the mean of the Gross Rent to the Standard Rent

library(dplyr)
library(ggpubr)

# filter the gross data from the main data and avoiding the null values and the values which are less than 0.
gross.rent <- mainData %>% select(GRNTP) %>% filter(GRNTP != "NA" , GRNTP != 0) %>% na.omit(month.rent)

# summary of the data
summary(gross.rent$GRNTP)

# plotting the histogram plot -to visualize the normality around the mean
ggplot(gross.rent, aes(x=GRNTP)) + 
 geom_histogram(aes(y=..density..), colour="black", fill="yellow" , alpha = 0.8)+
 geom_density(alpha=.2, fill="#FF6666" , color = 'green') + labs(title="Histogram for Gross Rent") +
  labs(x="Gross Rent", y="Desnity Values") + theme(plot.title = element_text(hjust = 0.5))  + geom_vline(aes(xintercept=mean(GRNTP)),
            color="blue", linetype="dashed", size=1)

# testing the normality here with qqplot
ggqqplot(gross.rent$GRNTP)


# for the large data set we use kolmogorov-smirnov test rather than shapiro wilkie test.
ks.test(gross.rent$GRNTP , y = 'pnorm',mean = 1070.964 , sd = 612.037)

 # performig the one sample T- test here.
t.test(gross.rent$GRNTP, mu = 1012)

Code-19 : Neural Network - Predicting the Value of the Property by Corresponding relative Predictors

library(neuralnet)
library(dplyr)
library(devtools)
source_url('https://gist.githubusercontent.com/fawda123/7471137/raw/466c1474d0a505ff044412703516c34f1a4684a5/nnet_plot_update.r')

nndata<- mainData %>% select(INSP,FINCP,FULP,GASP,VALP) %>% filter(FINCP != "NA" , INSP != "NA",FULP != "NA",GASP != "NA") %>% na.omit(nndata)

nndata <- head(nndata,1000)

# Random sampling
samplesizedata = 0.60 * nrow(nndata)
set.seed(80)
indexdatann = sample( seq_len ( nrow ( nndata ) ), size = samplesizedata )

# Create training and test set
datatrain = nndata[ indexdatann, ]
datatest = nndata[ -indexdatann, ]



## Scale data for neural network

maxnn = apply(nndata , 2 , max)
minnn = apply(nndata, 2 , min)
scaled = as.data.frame(scale(nndata, center = minnn, scale = maxnn - minnn))

trainNN = scaled[indexdatann , ]
testNN = scaled[-indexdatann , ]


set.seed(4)
NN = neuralnet(VALP ~ INSP + FINCP + FULP + GASP , trainNN, hidden = 3 , linear.output = T )

plot(NN , col.out = 'blue' , fontsize = 9,col.out.synapse = "red",col.intercept = "blue" , col.entry = 'skyblue',col.entry.synapse = 'red' , rep= "best")

Package Information/Installation

In order to perform the actual analysis and to create fancy graphs and tables I have used couple of new packages which are already not pre-defined and needs installation in most of the cases.

Please note that , All the packages I have used over here can easily be installed by one single command.

install.packages(“package name”)

Following are the packages you need to install:-

• 1. data.table
• 2. Knitr
• 3. pandar
• 4. skim
• 5. data.table
• 6. hrbrthemes
• 7. ggdark
• 8. ggpubr
• 9. viridis
• 10. plotly
• 11. caret
• 12. Kernlab
• 12. moments
• 13. kablextra (assuming that this is pre-installed)
• 13. Hmisc
• 13. corrplot
• 13. ellipse
• 14. RColorBrewer
• 15. mapproj
• 16. readr
• 17. maptools
• 18. rgdal
• 19. gpclib
• 20. maps
• 21. Dplyr (assuming that this is pre-installed)
• 22. Tidyverse (assuming that this is pre-installed)
• 23. broom
• 24. ggplot (assuming that this is pre-installed)
• 25. neuralnet